Handling rich turn-taking in spoken dialogue systems

This paper discusses how to build a system that can engage in a mixed-initiative human-machine spoken dialogue in which system utterances sometimes overlap with user utterances and vice versa. In the method, a module that incrementally understands user utterances and another module that incrementally generates system utterances work in parallel, and the timing of taking and releasing the dialogue initiative is decided according to the understanding of user utterances and the content of the system utterances. This method enables the system to respond when the user holds the dialogue initiative and is speaking, and enables the system to react to the user’s barge-ins when it holds the initiative and is speaking. An experimental system called DUG-1 is also presented.